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Background:  US  Air  Force  (USAF)  pilot  selection  and  training  procedures  have  changed 
substantially  in  the  last  several  years.  Pilot  selection  procedures  were  augmented  with  the 
introduction  of  computer-based  testing  to  assess  pilot  aptitude.  Training  procedures  have 
changed  in  an  effort  to  modernize  the  training  fleet  and  provide  better,  more  specialized  training 
earlier  in  the  training  process.  Results:  Despite  several  studies  showing  the  utility  of  USAF  pilot 
aptitude  tests  for  predicting  training  performance,  results  of  a  recent  policy  capturing  study 
suggest  that  this  information  is  often  ignored  by  pilot  candidate  selection  boards.  The  two  largest 
sources  of  USAF  pilot  trainees  relied  heavily  on  measures  of  officership  when  making  selection 
decisions.  Cumulative  research  findings  suggest  that  USAF  pilot  selection  decisions  could  be 
improved  by  making  better  use  of  currently  available  personnel  attribute  data.  Further 
improvements  could  be  expected  from  the  addition  of  a  structured  selection  interview  and 
measures  of  personality. 


THE  KEYWORD  IN  DESCRIBING  U.  S.  Air  Force  (USAF)  pilot  selection  and  training  over 
the  last  five  years  has  been  "change."  Training  procedures  have  changed  in  an  effort  to 
modernize  the  training  fleet  and  provide  better,  more  specialized  training  earlier  in  the  training 
cycle  (20).  Pilot  selection  procedures  have  changed  to  take  advantage  of  computer-based  testing. 
The  purpose  of  this  paper  is  to  review  these  changes. 

Changes  in  USAF  Pilot  Training 
Undergraduate  Pilot  Training  (UPT) 


In  Undergraduate  Pilot  Training  (UPT),  pilot  trainees  completed  primary  jet  training  in 
the  T-37  (21  weeks),  then  moved  on  to  advanced  jet  training  in  the  T-38  (32  weeks)  (see  Figure 
1).  Near  completion  of  T-38  training  an  advanced  training  recommendation  board  consisting  of 
T-38  instructor  pilots  evaluated  student  records  and  made  recommendations  for  post-UPT 
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specialized  training  assignments  in  either  tanker-transport-bomber  (TTB)  aircraft  or  fighter- 
attack-reconnaissance  (FAR)  air-craft.  Specialized  training  assignments  were  based  on  instructor 
recommendations,  student  preferences,  and  aircraft  availability. 


Undergraduate  Pilot  Training 


Specialized 

Training 


T-41  (FSP)  T-37  T-38 


Specialized  Undergraduate  Pilot  Training 


T-1A 


Figure  1.  UPT  and  SUPT  Pipelines 


Specialized  Undergraduate  Pilot  Training  (SUPT) 

UPT  was  phased  out  in  the  mid- 1 990’ s  and  replaced  with  Specialized  Under-graduate 
Pilot  Training  (SUPT).  Under  SUPT,  student  pilots  complete  a  common  T-37  training  phase  (as 
in  UPT).  At  the  end  of  T-37  training,  students  are  assigned  to  one  of  four  advanced  training 
tracks.  Most  are  assigned  to  either  the  bomber/fighter  (B/F)  or  airlift/tanker  (A/T)  tracks.  A  few 
are  assigned  to  the  T-44  advanced  turbo-prop  or  UH-1  helicopter  tracks.  Track  assignments  are 
based  on  T-37  training  performance,  aircraft  availability,  and  student  preference.  B/F  training 
(12)  takes  place  in  the  T-38  and  A/T  training  (13)  takes  place  in  the  T-1A,  a  modified  Beech  400 
business  jet.  The  T-l  has  an  impressive  electronic  flight  instrumentation  system  with  pilot 
configurable  multifunction  displays,  flight  management  system,  weather  radar,  and  electronic 
displays  of  maps  and  checklists. 

The  reasons  for  the  change  to  SUPT  included  the  desire  to  (a)  prolong  the  life  of  the  T- 
38,  (b)  upgrade  the  training  fleet,  and  (c)  improve  training.  As  not  all  students  would  need  to  go 
through  T-38  training  (as  in  UPT),  the  life  span  of  the  T-38  could  be  extended.  A  shortcoming  of 
the  generalized  flight  training  provided  in  UPT  was  that  it  did  not  provide  student  pilots  with 
specialized  skills  and  knowledge  needed  to  smoothly  transition  to  the  large  transport  and  tanker 
aircraft  that  constitute  one-third  of  the  USAF  fleet.  With  the  implementation  of  T-l  training,  A/T 
students  could  receive  specialized  instruction  at  an  earlier  stage  in  their  training 

Recently,  the  USAF  modified  the  training  sequence  for  bomber  pilots.  For  a  short  period 
starting  in  late  1998,  B-l  and  B-52  bomber  pilots  were  selected  from  among  T-l  trained  students 
effectively  making  the  T-38  track  a  "fighter  only"  track.  More  recently,  the  policy  has  changed 
again  where  B-l  pilots  are  selected  from  among  T-38  students  and  B-52  pilots  from  among  T-l 
students. 


More  Changes  to  Come 


The  Air  Force  goal  of  upgrading  the  trainer  fleet  with  current  technology  aircraft  has 
resulted  in  decisions  to  replace  the  aging  T-37  and  to  upgrade  T-38  avionics  to  contemporary 
standards.  The  aircraft  chosen  to  replace  the  T-37  is  the  Beech  Mk  II,  known  as  the  Texan  II;  2). 
Student  pilot  training  in  the  Texan  II  is  expected  to  begin  in  2001.  The  aircraft  will  eventually 
replace  all  T-37s  in  SUPT  and  Euro-NATO  Joint  Jet  Pilot  Training  (ENJJPT).  The  flight  training 
syllabus  for  the  Texan  II  will  resemble  that  used  today  for  T-37  primary  training,  similar  in 
duration,  flight  hours,  and  training  events  (3). 

Modifications  to  the  T-38  will  allow  the  USAF  to  include  mission  specific  weapons 
delivery  and  tactics  training  in  the  fighter  track.  A  major  avionics  upgrade  will  replace  existing 
T-38  instrumentation  commencing  in  1999  (1).  By  2008,  all  T-38A  and  T-38B  aircraft  used  in 
SUPT  and  ENJJPT  will  be  upgraded  to  T-38C  configuration. 

In  addition  to  the  T-38  hardware  upgrade,  current  plans  include  extending  fighter  track 
training  by  seven  weeks  to  incorporate  training  now  provided  in  a  separate  fighter  fundamentals 
course.  This  potential  change  is  not  likely  to  occur  until  the  T-38C  modification  has  been  fully 
integrated  into  the  force. 

Flight  Screening  Program 

The  pre-SUPT  Flight  Screening  Program  (FSP)  also  has  changed  significantly.  Prior  to 
1994,  FSP  consisted  of  about  14  hours  instruction  in  the  T-41.  In  1994,  the  T-41  was  replaced 
with  the  T-3  (Slingsby  Firefly)  and  the  hands-on  training  was  extended  to  21  hours.  This 
Enhanced  Flight  Screening  Program  (EFSP)  included  more  aerobatics  training  and  was  popular 
with  students  and  instructors.  However,  in  July  1997  the  T-3  flight  operations  were  suspended 
following  several  uncommanded  engine  stoppages  (4). 

With  the  suspension  of  T-3  training,  many  students  were  arriving  at  SUPT  with  no  hands- 
on  flying  experience.  This  placed  a  greater  burden  on  T-37  instructors  to  get  student  pilots  "up  to 
speed."  Even  students  who  had  prior  flying  time  in  commercial  aircraft  were  sometimes  having 
problems  adapting  to  military  flight  procedures.  Without  the  benefit  of  T-3  flight  screening, 
attrition  rates  for  SUPT  climbed  above  15  percent  (4).  The  USAF  considers  an  8  to  10  percent 
attrition  rate  acceptable. 

As  a  result  of  the  perceived  need  for  pre-SUPT  flying  training,  in  October  1998  the 
USAF  implemented  an  Introductory  Flight  Training  (IFT)  program.  The  IFT  program  included 
up  to  40  hours  flying  time  in  commercial  aviation  training  programs  and  required  a  solo  flight. 
The  40-hour  IFT  program  reportedly  has  produced  similar  attrition  rates  in  the  T-37  phase  of 
SUPT  as  were  observed  when  EFSP  was  in  use  (IFT  —  8.8%  vs.  EFSP  —  7.8%)  (4).  In  October 
1999,  the  USAF  announced  plans  to  implement  an  expanded  IFT  program  (4).  The  expanded 
program  will  increase  hands-on  flying  time  to  50  hours  and  will  require  additional  solo  flights 
and  students  to  earn  a  private  pilot's  license.  In  the  expanded  IFT  program,  students  will  receive 
Federal  Aviation  Administration-certified  flight  instruction  through  local  flight  schools.  More 
than  150  flight  schools  nationwide  may  be  involved  in  the  training  program,  mostly  near  colleges 
and  universities  with  Air  Force  Reserve  Officer  Training  Corps  (AFROTC)  detachments.  Pilot 
candidates  at  the  US  Air  Force  Academy  will  receive  their  training  in  the  local  area. 


After  completing  the  expanded  IFT  program,  pilot  candidates  will  enter  SUPT.  Although 


T-37  attrition  rates  have  been  similar  following  IFT  as  were  observed  following  EFSP,  some 
have  questioned  its  utility  for  USAF  training  and  screening.  Skeptics  note  that  IFT  is  not  done  in 
a  military  context  (as  was  EFSP)  and  emphasizes  training  only,  not  training  and  screening. 

USAF  Pilot  Selection  Methods 

US  Air  Force  pilot  qualification  standards  include  medical  fitness,  anthropometric 
standards,  educational  achievement  (e.g.,  college  grade  point  average,  major),  and  for  some 
commissioning  sources,  aptitude  test  scores  and  successful  performance  in  a  flight  screening 
program  (19).  The  indicators  of  pilot  aptitude  that  are  used  vary  by  source  of  commission. 
Further,  although  pilot  aptitude  test  scores  are  used  to  express  qualification  minimums  for  some 
points  of  entry,  the  manner  in  which  these  scores  are  used  by  selection  boards  varies  (18,  19). 


Pilot  Aptitude  Tests 


The  two  most  widely  used  pilot  aptitude  tests  in  the  USAF  are  the  Air  Force  Officer 
Qualifying  Test  (AFOQT;  9,  10)  and  the  Basic  Attributes  Test  (BAT;  6,  8).  The  AFOQT  is  a  16- 
test  paper- and- pencil  multiple  aptitude  battery  used  for  officer  commissioning  and  aircrew 
selection.  It  measures  general  cognitive  ability  (g),  verbal,  math,  spatial,  aviation  knowledge,  and 
perceptual  speed  (10).  The  AFOQT  has  been  in  use  since  1957  with  new  forms  developed  about 
every  seven  years.  The  16  tests  are  combined  into  five  composites:  Verbal,  Quantitative, 
Academic  Aptitude  (Verbal  +  Quantitative),  Pilot,  and  Navigator-Technical  (9,  10).  The  AFOQT 
Pilot  composite  consists  of  eight  tests  that  measure  knowledge  of  aviation  and  mechanical 
systems,  the  ability  to  determine  aircraft  attitude  from  flight  instruments,  knowledge  of 
aeronautical  concepts,  ability  to  read  and  interpret  scales  and  tables,  and  spatial  ability. 

The  BAT  is  a  five  test  computer-based  battery  used  solely  for  pilot  selection.  It  measures 
cognitive  ability,  psychomotor  ability,  and  attitudes  toward  risk.  The  BAT  was  operationally 
implemented  for  pilot  selection  in  1993.  Scores  from  the  BAT  are  combined  with  the  AFOQT 
Pilot  composite  and  a  measure  of  flying  experience  to  produce  a  pilot  aptitude  composite  known 
as  PCSM  (Pilot  Candidate  Selection  Method;  6,  8).  PCSM  scores  have  been  shown  to  be  related 
to  probability  of  completing  UPT  (6,7),  number  of  flying  hours  needed  to  complete  training  (11), 
class  ranking  (7),  and  fighter  qualification  (20).  Higher  PCSM  scores  are  associated  with  greater 
probability  of  completing  jet  training,  fewer  hours  needed  to  complete  training,  higher  class  rank, 
and  greater  likelihood  of  being  fighter  qualified. 


Table  1.  AFOQT  Minimum  Qualifying 
Scores  for  Pilot  Selection  by 
Commissioning  Source 


Minimum  AFOQT 

Composite  Scores 

Source 

P 

N 

P+N 

V 

Q 

OTS  w /  PPL 

25 

10 

50 

15 

10 

OTS  w/o  PPL 

50 

50 

60 

15 

10 

ROTC  (pre-1997) 

50 

10 

60 

15 

30 

ROTC  (1997  &  later)  25 

10 

50 

15 

10 

Active  Duty 

25 

10 

50 

* 

* 

AFA  (pie- 1998) 

* 

* 

* 

* 

* 

Notes.  "PPL”  means  private  pilot’s  license.  P  =  Pilot, 
N  =  Navigator-Technical,  V  =  Verbal,  and  Q  = 
quantitative. 

*  No  minimum  qualifying  score. 


Pilot  Selection  Standards 

As  illustrated  in  Table  1,  the  minimum  qualifying  AFOQT  scores  for  pilot  selection  vary 
by  commissioning  source.  The  strictest  standards  are  for  OTS  pilot  applicants  without  a  private 
pilot’s  license  (PPL).  The  weakest  standards  are  for  AFA  pilot  applicants.  The  AFA  sets  no 
minimum  qualifying  AFOQT  score,  instead  using  it  as  a  "tie  breaker"  when  evaluating  pilot 
applicants  (19).  There  are  no  established  minimum  qualifying  scores  for  the  BAT  or  PCSM. 

There  are  seven  major  points  of  entry  into  USAF  pilot  training  (19).  These  include  the 
Air  Force  Academy  (AFA),  Active  Duty  (AD),  Officer  Training  School  (OTS),  Reserve  Officer 
Training  Corps  (ROTC),  Air  National  Guard  (ANG),  Air  Force  Reserves,  and  international 
pilots.  This  paper  will  focus  on  the  first  four,  as  they  are  the  most  common  sources.  Weeks  and 
Zelenski  (19)  provide  a  more  detailed  description. 

Air  Force  Academy  (AFA).  The  AFA  requires  interested  cadets  to  complete  flight 
screening  for  qualification  prior  to  meeting  a  selection  board.  The  AFA  pilot  selection  board 
consists  of  six  or  seven  AFA  faculty  members  and  a  non-AFA  representative.  All  board 
members  are  pilots. 

Applicant  selection  folders  contain  (a)  the  Cadet  Administration  Management 
Information  System  (CAMIS)  sheet,  (b)  evaluations  by  AFA  faculty  members,  and  (c)  an 
optional  letter  by  the  pilot  candidate.  The  CAMIS  sheet  provides  detailed  information  on: 
personnel  data,  academic,  military,  and  athletic  performance,  honors,  flight  screening 
performance,  participation  in  airmanship  and  aviation  courses,  involvement  in  clubs,  probation 
history,  and  AFOQT  scores  (not  required  for  qualification).  No  explicit  gender  or  race/ethnicity 
information  is  provided. 

Active  Duty  (AD).  Active  duty  officers  may  apply  for  pilot  training  through  the  Air  Force 
Personnel  Center  (AFPC).  Qualification  is  based  on  age,  medical,  and  anthropometric  standards 
and  minimum  qualifying  AFOQT  scores.  The  AD  pilot  selection  board  consists  of  the  AFPC 
Vice  Commander,  an  AFPC  colonel,  and  two  lieutenant  colonels  that  are  current  or  former  flying 
squadron  commanders. 

The  AD  selection  board  process  is  similar  to  that  used  by  the  AFA  and  OTS.  Board 


members  are  instructed  to  use  a  "whole  person"  concept.  The  relative  importance  of  selection 
factors  is  not  prescribed,  but  is  left  up  to  the  individual  board  members.  Board  members  attempt 
to  reach  a  consensus  on  the  evaluation  of  each  record.  When  divergent  ratings  occur,  they  are 
resolved  by  rescoring  applicants  records. 

Application  folders  contain  an  Officer  Application  for  Flying  Training,  Officer 
Performance  Reports  (OPRs),  training  reports,  citations  accompanying  military  decorations,  a 
career  brief,  and  the  PCSM  score.  Unlike  the  AFA,  AD  selection  folders  contain  explicit 
information  about  applicants’  gender  and  race/ethnicity. 

Officer  Training  School  (OTS).  As  a  group,  OTS  pilot  applicants  tend  to  be  older  than 
other  USAF  pilot  applicants,  are  college  graduates,  and  may  have  significant  civilian  flying 
experience.  Many  have  prior  military  experience,  typically  as  active  duty  Air  Force  enlisted 
members.  OTS  pilot  selection  boards  consist  of  three  pilots  (colonels  or  colonel-selects). 

Compared  with  other  commissioning  sources,  OTS  pilot  applicant  folders  contain  the 
most  comprehensive  set  of  information.  As  with  the  AFA  and  Active  Duty,  board  members  are 
instructed  to  use  a  "whole  person"  approach.  Board  members  are  provided  a  scoring  policy  that 
suggests  the  relative  importance  of  three  broad  categories  of  information:  (a)  "experience" 
(civilian  or  military  employment,  outside  activities;  0-3  points),  (b)  "education  and  aptitude" 
(academic  history,  AFOQT,  PCSM;  0-3  points),  and  (c)  potential  and  adaptability" 
(recommendations;  0-4  pints).  As  with  Active  Duty  boards,  divergent  ratings  are  resolved  by 
rescoring  applicant  records. 

Selection  folders  include  aptitude  test  scores  (i.e.,  AFOQT  and  PCSM),  biographical  and 
educational  information,  employment  history,  history  of  legal  violations,  civilian  flying  records 
including  flying  and  FAA  medical  certificates,  letters  of  recommendation,  and  other  data.  As 
with  Active  Duty  selection  procedures,  OTS  selection  boards  are  provided  explicit  information 
about  candidates’  gender  and  race/ethnicity. 

Reserve  Officer  Training  Corps  (ROTC).  ROTC  pilot  applicants  are  cadets  in  their 
junior  year  of  college.  There  are  nearly  150  college  and  university  ROTC  detachments  that  can 
provide  applicants.  A  centralized  board  consisting  of  rated  field  grade  officers  from  HQ 
AFROTC  at  Maxwell  AFB  makes  pilot  candidate  selection  decisions. 

The  process  of  categorizing  ROTC  pilot  applicants  is  different  from  that  used  by  the 
other  commissioning  sources.  Board  members  are  not  required  to  manually  review  the  detailed 
contents  of  folders.  The  ROTC  pilot  selection  process  is  supported  by  preprocessing  of  applicant 
data  to  create  a  preliminary  rank-order  of  applicants.  As  a  result,  the  workload  for  ROTC  board 
members  is  minimized  and  the  relative  importance  of  selection  factors  is  the  same  for  all 
applicants. 

The  algorithm  that  supports  ROTC  pilot  candidate  selection  weights  selection  factors  to 
create  a  composite  known  as  the  Categorization  Order  of  Merit  (COM)  score.  The  information 
that  contributes  to  the  COM  score  includes  an  evaluation  of  the  cadet’s  officer  potential  by  the 
detachment  commander  (i.e.,  Relative  Standing  Score),  college  grade  point  average,  AFOQT 
composite  scores,  and  a  physical  fitness  test  score.  It  is  interesting  to  note  that  prior  civilian 
flying  experience  does  not  contribute  to  the  COM  score.  Prior  to  1998,  the  PCSM  score  also  did 


not  contribute  to  the  COM  score. 


Cadets  are  rank-ordered  from  highest  to  lowest  based  solely  on  COM  score.  The  pilot 
production  requirement  is  applied  to  produce  an  initial  cut  line.  The  selection  board  then  reviews 
the  list  considering  all  information  presented.  It  may  accept  the  rank-ordered  list  as  presented  or 
adjust  the  ordering  on  the  basis  of  selection  criteria  independent  of  the  COM  score  (e.g.,  ROTC 
detachment,  gender,  race/ethnicity,  previous  flying  experience,  field  training  awards,  academic 
major). 

Selection  Ratios 

The  proportion  of  applicants  selected  for  pilot  training  varies  by  point  of  entry  and  as  a 
function  of  the  needs  of  the  US  Air  Force  (i.e.,  annual  pilot  production  quotas).  Over  the  last 
several  years,  most  pilot  candidates  have  come  from  the  AFA  and  ROTC.  In  contrast,  OTS 
selection  boards  have  been  very  selective  during  a  period  of  low  pilot  production.  For  example, 
in  1996  the  pilot  selection  ratios  were  .789  (375/475)  for  the  AFA,  .147  (81/551)  for  Active 
Duty,  .047  (16/340)  for  OTS,  and  .711  (382/537)  for  ROTC  (16).  The  combined  selection  ratio 
for  all  four  sources  was  .449  (854/1,903). 

USAF  Pilot  Selection  Board  Policy  Capturing  Study 

The  USAF  recently  conducted  a  policy  capturing  exercise  to  better  understand  the  process  by 
which  the  AFA,  Active  Duty,  OTS,  and  ROTC  pilot  selection  boards  made 

their  decisions  (18,  19).  Policy  capturing  involves  quantifying  the  process  used  by  a  decision¬ 
maker  through  which  informational 

attributes  are  weighed  and  combined  to  make  a  decision.  Results  for  Active  Duty  and  OTS 
boards  should  be  viewed  as  preliminary  due  to  the  small  number  of  pilots  produced  from  those 
commissioning  sources. 

The  policy  capturing  method  used  was  the  same  for  each  point  of  entry.  The  only 
differences  were  in  the  information  available  in  the  applicants'  folders  and  the  timeframe  for  the 
data  collection.  So,  rather  than  describing  the  policy  capturing  method  separately  for  each,  it  is 
described  only  for  the  AFA.  The  AFA  provided  applicant  folders  and  average  board  ratings  for 
the  1995,  1996,  and  1997  classes.  Research  team  members  observed  the  selection  board  process 
and  surveyed  board  members  to  identify  the  most  important  attributes.  A  database  was 
constructed  and  a  series  of  linear  regression  analyses  were  performed  to  determine  which 
applicant  variables  were  actually  used  by  selection  boards.  Applicant  information  was  used  as 
the  independent  variables  and  average  board  rating  was  the  dependent  variable.  Using  input  from 
selection  board  members,  relevant  applicant  attributes  were  selected  and  included  in  a 
preliminary  "baseline"  selection  policy  model.  Variables  that  were  highly  significantly  correlated 
with  average  board  rating  were  retained  and  those  that  were  not  were  discarded.  Once  a  model 
was  determined  for  a  specific  board  for  a  specific  year,  it  was  applied  to  the  following  yearly 
boards  to  evaluate  the  stability  of  the  policy  model  across  boards.  For  the  AFA,  selection  policy 
models  were  developed  for  1995,  1996,  and  1997,  and  a  general  model  was  developed  for  all 
years  combined. 

The  yearly  AFA  pilot  selection  board  policy  capturing  models  showed  great  consistency 


from  year  to  year,  with  no  statistically  significant  differences  between  the  separate  yearly 
models.  The  general  model  (across  years)  showed  a  very  good  fit,  accounting  for  90%  of  the 
variability  in  board  ratings  (R2  =  .90).  Results  indicated  a  heavy  reliance  on  cumulative  military 
performance  average,  cumulative  grade  point  average,  and  flight  screening  performance. 
Variables  that  contributed  to  a  lesser  extent  to  the  AFA  policy  capturing  model  were: 
participation  in  a  varsity  sport,  whether  or  not  the  applicant  had  held  a  commander  military 
position,  and  whether  or  not  the  cadet  had  held  a  lower  military  position  in  their  senior  year. 
"Officership"  as  indicated  by  military  performance  was  more  important  for  pilot  selection 
decisions  than  was  ability  as  measured  by  cumulative  grade  point  average  and  flight  screening 
performance  rating.  AFOQT  scores  were  given  little  importance  and  were  used  mainly  as  a  "tie 
breaker"  for  distinguishing  among  candidates  close  to  the  "cut  line." 

For  Active  Duty  pilot  selection  boards,  separate  policy  capturing  analyses  were  done  for 
non-rated  line  officers  and  navigators  for  fiscal  years  1996  and  1997.  In  each  case,  the  resultant 
general  (combined  fiscal  years)  policy  capturing  model  showed  a  good  fit  (R2  =  .78  and  .72 
respectively).  For  non-rated  line  officers,  the  most  important  selection  factor  by  far  was  PCSM. 
Variables  that  contributed  to  a  lesser  extent  to  the  non-rated  model  were:  2nd  lieutenant  rank, 
positive  endorser's  recommendation,  possession  of  an  engineering  or  mathematics  degree, 
possession  of  a  master's  degree,  and  possession  of  a  flying  instrument's  rating. 

For  Active  Duty  navigators,  the  most  important  selection  factors  were  PCSM  and  the 
AFOQT  Quantitative  composite.  Positive  endorser's  recommendation  and  positive  commander's 
recommendation  contributed  to  a  lesser  extent. 

OTS  selection  boards  provided  applicant  folders  and  average  board  ratings  for  the  first 
and  second  quarters  of  fiscal  year  1996.  The  general  (combined  quarters)  OTS  pilot  selection 
board  policy  capturing  model  showed  a  relatively  poor  fit  (R2  =  .51)  compared  with  results  from 
the  other  points  of  entry.  The  relatively  poor  fit  was  probably  due  to  the  small  number  of 
applicants  selected.  The  most  important  factors  were  college  grade  point  average  and  PCSM 
score.  Number  of  recommendation  letters,  interviewers’  recommendations,  number  of  moving 
traffic  violations,  and  possession  of  a  bachelor  of  arts  degree  contributed  to  a  much  lesser  extent 
to  the  OTS  policy  capturing  model. 

For  the  ROTC,  the  COM  score  was  by  far  the  most  important  factor  in  pilot  candidate 
selection.  Very  few  candidates  were  re-ordered  on  the  basis  of  non-COM  data.  As  a  result,  no 
formal  policy  capturing  study  was  conducted  (18,  19).  Instead,  analyses  focused  on 
understanding  the  relative  importance  of  the  COM  components  in  determining  the  overall  COM 
score.  Examination  of  the  weights  and  upper  and  lower  bounds  of  the  COM  components 
revealed  that  the  commander’s  assessment  of  the  cadet’s  officer  potential  (Relative  Standing 
Score  or  RSS)  accounted  for  about  47.8%  of  the  variability  in  the  COM  score.  The  other  COM 
components  accounted  for  the  remaining  52.2%  as  follows:  college  grade  point  average  (19.8%), 
physical  fitness  test  score  (11.5%),  AFOQT  Verbal  composite  (11.5%),  and  AFOQT 
Quantitative  composite  (9.4%). 

Examination  of  Pearson  correlation  coefficients  between  the  COM  score  and  its 
components  provided  another  way  of  looking  at  the  relative  importance  of  the  COM  components 
(18,  19)  in  determining  overall  COM  score.  As  in  the  analysis  of  the  COM  weights,  RSS  had  the 
strongest  relationship  with  overall  COM  score.  The  correlations  between  COM  and  its 


components  were:  RSS  (.921),  college  grade  point  average  (.639),  physical  fitness  test  score 
(.437),  AFOQT  Verbal  composite  (.290),  and  AFOQT  Quantitative  composite  (.325).  These 
results  taken  together  clearly  showed  that  RSS  was  the  most  influential  determinant  of  the 
overall  COM  score,  and  as  a  result,  ROTC  pilot  selection  decisions.  Pilot  aptitude  measures  such 
as  PCSM  or  previous  flying  experience  were  virtually  ignored  in  the  ROTC  pilot  selection 
process. 

The  two  largest  sources  of  USAF  pilots  (AFA  and  ROTC)  place  a  much  greater  emphasis 
on  measures  of  officership  (AFA  -  military  performance  average,  ROTC  -  RSS)  than  measures 
of  ability  (e.g.,  academic  performance,  flight  screening  performance,  aptitude  test  scores)  when 
selecting  pilot  trainees.  This  is  rather  perplexing  in  the  absence  of  studies  showing  validity  for 
measures  of  officership  for  predicting  pilot  training  performance. 

Weeks  (18)  reported  a  study  of  the  relative  predictive  utility  of  officership  as  measured 
by  the  RSS  and  ability  as  measured  by  the  AFOQT  Pilot  composite  for  469  ROTC  cadets  who 
attended  undergraduate  pilot  training.  The  correlation  between  RSS  and  pass/fail  training  was  .01 
(ns),  whereas  the  correlation  between  AFOQT  pilot  composite  and  pass/fail  training  was  .14  (p  < 
.01).  These  analyses  indicate  almost  no  relationship  between  officership  ratings  and  pilot  training 
attrition.  The  strength  of  the  relationship  between  ability  and  pilot  training  outcome  is  lower  than 
usually  reported  (see  for  example,  8,  9),  but  still  clearly  shows  that  ability  represents  an 
improvement  over  officership  ratings  for  selection. 

Validity  of  PCSM  for  T-37  Training 

Operational  PCSM  testing  began  in  June  1993  and  selection  boards  first  received  PCSM 
scores  in  December  1993.  Air  Education  and  Training  Command  Studies  and  Analyses  Squadron 
(AETC/SAS)  subsequently  conducted  an  evaluation  of 

PCSM  in  June  1997  (16).  At  the  time  of  the  evaluation,  7,057  USAF  pilot  training  applicants  had 
valid  PCSM  scores  (had  tested  on  the  AFOQT  and  BAT).  Of  these,  1,276  had  been  selected  and 
entered  pilot  training;  676  either  had  been  eliminated  from  training  or  had  successfully 
completed  T-37  training.  The  T-37  graduation  rate  was  81.2%  (549/676).  Ness  (16)  did  not 
report  the  correlation  between  PCSM  and  the  dichotomous  pass/fail  training  outcome,  but 
reported  the  pass  rate  for  trainees  grouped  by  PCSM  score  (1-25,  26-50,  51-76,  and  76-99).  The 
T-37  pass  rates  for  the  four  groups  were:  1-25:  58.4%  (66/113);  26-50:  69.7%  (83/119);  51-75: 
82.4%  (103/125);  and  76-99:  93.1%  (297/319).  Ness  (16)  observed  that  pilot  trainees  with  PCSM 
scores  of  50  or  lower  were  much  less  likely  to  complete  T-37  training  than  were  those  with 
scores  greater  than  50  (64.2%  vs.  90.1%  pass  rate).  Based  on  these  results,  he  recommended  that 
a  minimum  PCSM  score  of  50  be  adopted  and  that  PCSM  be  extended  to  the  AFA. 

In  a  further  breakout  of  PCSM  scores  for  the  676  pilot  trainees,  Ness  (16)  displayed  the 
number  of  trainees  with  scores  of  1-25,  26-50,  51-75,  and  76-99  for  each  commissioning  source 
that  tested  their  pilot  applicants  on  PCSM  (AD,  OTS,  and  ROTC).  Almost  all  AD  and  OTS  pilot 
trainees  had  PCSM  scores  greater  than  50,  with  most  scores  greater  than  75.  In  contrast,  ROTC 
pilot  trainees  were  nearly  equally  likely  to  have  scores  in  any  of  the  four  PCSM  groups  (1-25, 
26-50,  51-75,  and  76-99).  This  is  consistent  with  the  pilot  selection  board  policy  capturing  study 
results  (18,  19)  that  AD  and  OTS  selection  boards  relied  on  PCSM  in  their  selection  process  and 
ROTC  boards  did  not. 


A  second  examination  of  the  utility  of  PCSM  was  conducted  for  this  paper  in  order  to  get 
an  estimate  of  its  validity  against  T-37  outcome.  As  of  Fall  1998,  1,268  USAF  pilot  trainees  with 
PCSM  scores  had  completed  the  T-37  phase  of  SUPT.  The  T-37  graduation  rate  was  80.1% 
(1,016/1,268;  see  footnote  6).  As  observed  in  the  AETC/SAS  PCSM  evaluation  (16), 
examination  of  the  PCSM  scores  for  these  trainees  clearly  showed  that  some  selection  boards  did 
not  use  the  PCSM  score.  The  mean  PCSM  percentile  score  was  62.8  and  ranged  from  1  to  99. 
Two  hundred  twenty-nine  (18.1%)  of  the  trainees  had  PCSM  scores  of  25  or  lower.  Twenty-three 
(1.8%)  had  PCSM  scores  of  1. 
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The  observed  correlation  between  PCSM  and  the  dichotomous  pass/fail  T-37  training 
criterion  was  .335.  The  correlation  increased  to  .479  after  correction  for  dichotomization  of  the 
training  criterion.  As  shown  in  Figure  2,  higher  PCSM  scores  were  clearly  related  to  greater 
probability  of  successfully  completing  T-37  training.  The  pass  rates  for  the  four  groups  were:  1- 
25:  58.5%  (134/229);  26-50:  69.7%  (163/234);  51-75:  81.8%  (180/220);  and  76-99:  92.1% 
(539/585).  As  noted  earlier,  AD,  OTS,  and  ROTC  currently  set  minimum  qualifying  AFOQT 
Pilot  composite  scores  for  pilot  training  with  values  of  either  25  or  50  (see  Table  1).  If  a 
minimum  qualifying  PCSM  score  of  25  had  been  used,  the  overall  T-37  training  pass  rate  would 
have  been  about  84.5%  (882/1,039)  instead  of  the  observed  80.1%  (1,016/1,278).  Had  a 
minimum  qualifying  PCSM  score  of  50  been  used,  as  suggested  by  Ness  (16),  the  T-37  training 
pass  rate  would  have  been  about  89.3%  (719/805).  Results  from  both  Ness  (16)  and  the  current 
study  clearly  show  that  establishment  of  a  minimum  PCSM  score  consistent  with  current 
AFOQT  minimum  scores  for  pilot  training  qualification  could  help  reduce  training  attrition. 

Final  Thoughts 


Current  AETC  efforts  are  focusing  on  upgrading  the  BAT  hardware  and  test 
battery.  AETC  has  begun  the  work  needed  to  replace  the  original  386-based  systems  with 
Pentium-based  systems  and  to  evaluate  candidate  replacement  BAT  tests. 

Cumulative  research  findings  suggest  that  USAF  pilot  selection  decisions  could 
be  improved  simply  by  making  better  use  of  currently  available  personnel  attribute  data.  Higher 
PCSM  scores  are  associated  with  greater  probability  of  completing  jet  training  (6,  7,  16),  fewer 
flying  hours  needed  to  complete  training  (11),  higher  class  rank  (7),  and  greater  likelihood  of 
being  fighter  qualified  (20).  Despite  this,  to  date  no  minimum  qualifying  PCSM  score  has  been 


adopted.  The  reason  for  this  is  unclear.  It  appears  that  the  various  commissioning  sources  are 
reluctant  to  give  up  any  autonomy  they  currently  have  in  setting  pilot  selection  policy.  However, 
the  PCSM  program  has  been  extended  till  2003.  AETC/CC  has  directed  the  use  of  PCSM  in  all 
pilot  candidate  boards.  As  of  March  1999,  all  have  implemented  PCSM  except  the  AFA.  In 
1998,  ROTC  modified  the  COM  equation  removing  the  AFOQT  Pilot  composite  and  replacing  it 
with  the  PCSM  score  (14).  The  effectiveness  of  this  modification  for  reducing  pilot  training 
attrition  will  depend  on  the  relative  weight  assigned  to  the  new  COM  components. 

Results  from  recent  studies  in  personnel  selection  and  occupational  performance  provide 
some  clues  as  to  how  personnel  attribute  data  can  be  better  integrated  into  the  pilot  selection 
process  (8,  15,  17).  The  validity  of  PCSM  has  been  shown  to  come  mostly  from  the 
measurement  of  cognitive  ability,  psychomotor  ability,  pilot  job  knowledge,  and  flying  (i.e.,  job- 
related)  experience  (8).  This  is  consistent  with  results  from  a  large-scale  meta-analysis  of  the 
validity  and  utility  of  selection  methods  for  training  and  job  performance  across  a  variety  of 
occupations  (17).  Schmidt  and  Hunter  (17)  estimated  the  predictive  validity  of  general  mental 
ability  (GMA)  tests  to  be  .56  for  training.  The  two  combinations  of  predictors  with  the  highest 
multivariate  validity  for  job  training  programs  were  GMA  plus  an  integrity  test  (mean  validity  of 
.67)  and  GMA  plus  a  conscientiousness  test  (mean  validity  of  .65).  For  job  performance,  the 
predictive  validity  of  GMA  tests  was  estimated  to  be  .51  and  was  incremented  by  the  addition  of 
a  work  sample  test  (.63),  a  structured  interview  (.63),  an  integrity  test  (.65),  or  a  conscientious¬ 
ness  test  (.60).  Similarly,  McHenry,  Hough,  Toquam,  Hanson,  and  Ashworth  (15)  reported  that 
measures  of  temperament/personality  were  incremental  to  cognitive  measures  for  predicting  US 
Army  training  criteria  reflecting  "effort  and  leadership,"  "personal  discipline,"  and  "physical 
fitness  and  military  bearing."  McHenry  et  al.'s  (15)  results  suggest  that  measures  of 
temperament/personality  might  be  useful  for  officer  selection,  which  is  closely  tied  to  pilot 
selection  in  the  USAF,  as  pilot  applicants  must  first  be  accepted  for  officer  commissioning  (19). 

To  reiterate,  the  effective  use  of  currently  available  personnel  attribute  data  could  lead  to 
cost  avoidance  savings  due  to  reduced  training  attrition  (16)  and  the  requirement  for  fewer 
hands-on  flying  training  hours  (11).  Results  from  studies  of  personnel  selection  and  occupational 
performance  (15,  17)  suggest  that  the  validity  and  utility  of  current  USAF  pilot  selection 
methods  (i.e.,  selection  boards,  PCSM)  could  be  enhanced  by  the  introduction  of  a  structured 
selection  interview  and  measures  of  personality. 
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